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In the Claims: 



11. A method forWiotating video, comprising the steps of: 
receiving video, said video depicts a surface at said live event; 

receiving a graphic manually created by a human operator during said live event; and 
blending said graphic with said video such that said graphic appears to be drawn on said 
surface, said step of blending being performed during said live event. 



12. A method according to J^laim 1 1, wherein: 
said graphic includes a curve. 

13. A method according to claimu 1, wherein: 
said step of receiving includes receiving a curve drawn on a touch screen. 



14. A method according to claim 11, wherein: 
said step of blending includes determining a set of Wending values and transmitting said blending 
values to a keyer. 



15. A method according to claim 1 1, wherein:\ 

prior to said step of blending, at least a portion of said video depicts at least a portion of said 
surface at a live event and a set of one or more objects occluding said surface; and 

said step of blending includes blending said graphic with siaid portion of said video without 
drawing said graphic over said objects. 
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16. A method according to claim 11, wherein: 

said step of receiving a graphic includes receiving two dimensional position information for said 
graphic created in relation to a\wo dimensional image. 



17. A method according to claim 16, further comprising the steps of: 

converting a first set of one or mbre two dimensional positions to one or more three dimensional 

locations, said first set of one or more tw^ dimensional positions correspond to said two dimensional 

position information for said graphic; 

converting said one or more three dlunensional locations to a second set of one or more two 

dimensional positions, said step of blending includes blending said graphic with said video based on said 

second set of one or more two dimensional positions. 



18. A method according to claim 17,Yurther comprising the steps of: 
receiving camera sensor data for a first cameraisaid video being captured by said first camera, 
said step of converting said one or more three dimensional locations to a second set of one or more two 
dimensional positions is performed uses said camera Censor data for said first camera. 



19. A method according to claim 18, wherein^ 



said step of receiving camera sensor data includes receiving information from an inclinometer. 



20. A method according to claim 18, wherein: 
said step of receiving camera sensor data includes receiving information from a gyro. 
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21. A method according to claim 18, further comprising the steps of: 

receiving camera sensor aata for a second camera, said two dimensional image being captured 
by said second camera, said step o£ converting said first set of one or more two dimensional positions 
to one or more three dimensional locations uses said camera sensor data for said second camera. 

22. A method accordin&to claim 17, further comprising the steps of: 

receiving camera sensor data for a first camera, said two dimensional image being captured by 
said first camera, said step of converting said first set of one or more two dimensional positions to one 
or more three dimensional locations uses \aid camera sensor data for said first camera. 

23. A method according to claim\l7, wherein: 

said two dimensional position information is different than said first set of one or more two 
dimensional positions. 



24. A method according to claim 17, further comprising the steps of: 

dividing said graphic into segments based on said two dimensional position information, said 

first set of one or more two dimensional positions are end^oints of said segments; and 

thickening said graphic after said step of converting a first set of one or more two dimensional 

positions to one or more three dimensional locations. 



25. A method according to claim 17, wherein: 
said two dimensional image is part of a first video frame; 
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said step of blending includes blending said graphic with a second video frame different than 
said first video frame. 



26. A method according to claim 17, wherein: 

said two dimensional imagers part of a first video frame from a first camera; and 



said step of blending includes blending said graphic with a second video frame from a second 



camera. 



27. A method according to claim 17, wherein: 

said first set of one or more two dimensional positions pertains to only a new portion of said 
graphic; and 

said one or more three dimensional locations pertain to said graphic in its entirety. 



28. A method according to claim 17, further^comprising the step of: 

creating a model of at least a portion of said surface, said step of converting said first set of one 

or more two dimensional positions to one or more three dimensional locations is performed using said 

model. 



29. A method according to claim 1 1, further comprising the step of: 
smoothing said graphic prior to said step of blending, said graphic is a drawing. 



30. A method according to claim 1 1, wherein: 

said step of blending includes performing a flicker filter. 



5- 
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31. A method according to claim 11, wherein: 
said video is captured bMa first camera; 
said graphic includes a ciirve; 

said curve is represented as a set of quadrilaterals; and 
said step of blending includes tessellating said quadrilaterals if said first camera has been 
zoomed past a threshold. 



32. A method according to blaim 11, wherein said step of blending includes: 
adding said graphic to a first vid\p image at a first position in said first video image; 
adding said graphic to a second video image at a second position in said second video image; 
said first position is different than said second position; 

said first position corresponds to a position of an image of a first portion of said surface in said 

first video image; and 

said second position corresponds to a position of said image of said first portion of said surface 

in said second video image. 



33. A method for annotating video, comprising the steps of: 

receiving video, at least a portion of said video depictsat least a portion of a surface at a live 

event and a set of one or more objects occluding said surface^ 

receiving a graphic manually created by a human operator during said live event; and 
blending a graphic with said portion of said video withourdrawing said graphic over said 

objects, said steps of receiving video, receiving a graphic and blending are performed during said live 

event. 
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34 . A method according to 
accessing color data for said v 



> ;laim 33, wherein said step of blending includes the steps of: 
deo; 

using said color data to access stored blending values, said stored blending values are organized 
by colors; and 

causing a blending of said graphic^ with said video based on using said blending values. 



35. A method according to claibi 34, further comprising the steps of: 

receiving a selection of a portion of an image, said portion of said image includes a set of colors; 
receiving said blending values for said set of colors; and 
storing said blending values. 

36. A method according to claim 34,\wherein: 
said step of using includes the steps of: 

comparing a first color value for a firkt pixel in said video to a color map, said color 
map stores color values and corresponding blending values, and 

identifying a first blending value corresponding to said first color value based on said 

color map; and 

said step of causing a blending includes using sai& first blending value as part of a process to 
blend said first pixel with a corresponding pixel in saicigraphic. 

37. A method according to claim 33, whereinlc 
said step of receiving a graphic includes receiving two dimensional position information for said 

graphic created in relation to a two dimensional image. 
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38. A method accordingtto claim 37, further comprising the steps of: 
converting a first set of one or more two dimensional positions to one or more three dimensional 

locations, said first set of one or more twd dimensional positions are based on said two dimensional 
position information for said graphic; and 

converting said one or more three dimensional locations to a second set of one or more two 
dimensional positions, said step of blending includes blending said graphic with said video based on said 
second set of one or more two dimensional positions. 

39. A method according to claimv38, wherein: 

said two dimensional image is part of a first video frame; and 

said step of blending includes blending saia graphic with a second video frame different from 
said first video frame. \ 

40. A method according to claim 33, farther comprising the step of: 
smoothing said graphic prior to said step of mending, said graphic is a drawing. 

41. A method according to claim 33, wheiein said step of blending includes: 
adding said graphic to a first video image at a first position in said first video image; 
adding said graphic to a second video image at a sec&nd position in said second video image; 
said first position is different than said second position; 

said first position corresponds to a position of an imageW a first portion of said surface in said 
first video image; and \ . 
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said second position corresponds to a position of said image of said first portion of said surface 
in said second video image. 

42. A method for annotating video, comprising the steps of: 

receiving video, said video uepicts a surface at said live event; 

receiving two dimensional position information for at least a portion of a graphic created in 
relation to a two dimensional image; 

converting a first set of one or more\two dimensional positions to one or more three dimensional 
locations, said one or more two dimension^ positions correspond to said two dimensional position 
information; 

converting said one or more three diniensional locations to a second set of one or more two 
dimensional positions; and 

blending said graphic with said video basdd on said second set of one or more two dimensional 

positions. 

43. A method according to claim 42, farther comprising the steps of: 
receiving camera sensor data for a first camera, Said video being captured by said first camera, 
said step of converting said one or more three dimensionaMocations to a second set of one or more two 
dimensional positions is performed uses said camera sensor data for said first camera. 



44. A method according to claim 43, further comprising the steps of: 
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receiving camera sensor Idata for a second camera, said two dimensional image being captured 
by said second camera, said step of converting said first set of one or more two dimensional positions 
to one or more three dimensional locations uses said camera sensor data for said second camera. 

45. A method according to claim 42, further comprising the steps of: 

receiving camera sensor data for a first camera, said two dimensional image being captured by 
said first camera, said step of converting^aid first set of one or more two dimensional positions to one 
or more three dimensional locations uses said camera data for said first camera. 

46. A method according to claim 42, wherein: 

said two dimensional position information is different than said first set of one or more two 
dimensional positions. 

47. A method according to claim 42, further comprising the steps of: 
dividing said graphic into segments basedW said two dimensional position information, said 

first set of one or more two dimensional positions are end points of said segments; and 

thickening said graphic after said step of converting a first set of one or more two dimensional 
positions to one or more three dimensional locations. 

48. A method according to claim 42, whdrein: 
said two dimensional image is part of a first vitieo frame; and 

said step of blending includes blending said graphi^ with a second video frame different from 
said first video frame. 



10 
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49. A method according to claim 42, wherein: 

said two dimensional image is i>art of a first video frame from a first camera; and 
said step of blending includes blei iding said graphic with a second video frame from a second 



camera. 



50. A method according td claim 42, wherein: 

said first set of one or more two dimensional positions pertains to only a new portion of said 
graphic; and 

said one or more three dimensional locations pertain to said graphic as a whole. 



51. A method according to 
creating a three dimensional modd 



:laim 42, further comprising the step of: 
of at least a portion of said surface, said step of converting 



said first set of one or more two dimensional positions to one or more three dimensional locations uses 
said model. 



52. A method according to claim 42, further comprising the step of: 
smoothing said graphic prior to said step of blending, said graphic is a drawing. 



event; 



53. A method for annotating 4ideo, comprising the steps of: 
receiving video, said video depicts a surface at said live event; 
receiving at least a portion of a drawing manually created by a human operator during said live 

smoothing said drawing; and 
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blending said smoothed drawing with said video during said live event. 

54. A method accordingVo claim 53, further comprising the step of: 
thickening said drawing. \ 

55. A method according to claim 53, wherein said step of smoothing said drawing includes 
the steps of: \ 

receiving a new point; \ 

adding said new point to a control group if there are is a sufficient distance between said new 
point and another point in said control group ;\ 

drawing a line between said new point ana said another point in said control group if there is 
not more than two points in said control group; and 

removing a next to last point in said control group and fitting a Bezier spline through points in 
said control group, if there are more than two points in said control group. 

56. A method for blending images, comprising the steps of: 
storing blending values for a set of colors; \ 

receiving a first image after said step of storing* 
receiving a second image after said step of storirk; 
accessing color data for said second image; \ 

comparing said color data to said blending values rbr said set of colors; and 
causing a blending of said first image with said second image based on using said blending 
values for said set of colors. \ 

- 12- \ 
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57. A method according to claim 56, further comprising the steps of: 
receiving a selection of a pprtion of a third image, said portion of said third image includes a 

subset of said set of colors; and 

receiving one or more blending values for said subset of said set of colors. 

58. A method according to\claim 57, wherein: 
said second image is part of a viaeo of an event; and 
said first image is an effect to be kdded to said video of said event. 

59. A method according to clairiji 58, wherein: 
said video is live video; and 

said step of receiving one or more blendin^values includes receiving a number from an operator 
indicating how to blend said first image with said second image. 

60. A method according to claim 56, wherein: 
said blending values are alpha percentages. 

61. A method according to claim 56, whetfpin: 
said step of comparing includes the steps of: 

comparing a first color value for a first pixel in said second image to a color map, said 
color map stores color values for said set of colors and said blinding values for said set of colors, and 
identifying a first blending value corresponding to said first color value based on said 

color map; and 
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said step of causing a blenliing includes using said first blending value as part of a process to 
blend said first pixel with a corresponding pixel in said first image. 

62. A method according to claim 56, wherein: 
said steps of accessing and comparing include the steps of: 

accessing a first pixel i(n said second image, said first pixel having a first color value, 
accessing additional pixel^ nearby to said first pixel in said second image, said additional 
pixels having additional color values, 

comparing a first color valuk for said first pixel to a color map, said color map stores 
color values for said set of colors and said Wending values for said set of colors, 
comparing said additional color values to said color map, 
identifying a first blending valuAcorresponding to said first color value based on said 



color map, 



identifying additional blending corresponding to said additional color values based on 



said color map, 



calculating an average of said first blending value and said additional blending values, 



and 



adjusting said average based on a weighting factor; and 
said step of causing attending includes using saidWdjusted average to blend said first pixel with 
a corresponding pixel in said first image. 

63. A method according to claim 56, whereii 
said step of causing a blending includes performing a flicker filter. 
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64. A method according to claim 56, wherein: 

said set of colors includes aVirst subset of colors and a second subset of colors; 

said first subset of colors is associated with a first blending value; and 

said second subset of colors represents a taper zone having blending values ranging from at or 

near said first blending value toward a second blending value, said second blending value represents 

no blending of said first image with saidlsecond image. 

65. A method according to claim 56, wherein: 
set of colors includes multiple visibly distinct colors. 

66. A method according to claim\56, wherein: 

said step of accessing color data includes Accessing color data for a first portion of said second 
image, said first portion of said image corresponds in position to said first image; and 

said step of causing a blending includes causing said first image to be blended with said first 
portion of said second image and not other portions of said second image. 

67. A method according to claim 56, wherein: 

said second image is a live video image of a live event; and 

said steps of storing, receiving a first image, receiving a second image, accessing, comparing 
and causing are performed during said live event. 

68. A method according claim .67, further including the step of: 
updating said blending values for said set of colors Wring said live event. 
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69. An apparatus, comprising: 
one or more processors; 

an input device in communic&ion with said one or more processors; 
an output device in communication with said one or more processors; and 
at least one storage device in communication with said one or more processors, said processors 
programmed to preform a method comprising the steps of: 

storing blending values fc^r a set of colors, 

receiving a first image, 

receiving a second image, 

accessing color data for said second image, 

comparing said color data tolsaid blending values for said set of colors, and 
causing a blending of said first image with said second image based on using said 
blending values for said set of colors. 

70. An apparatus according to claim p9, wherein said method further includes the steps 

of: 

receiving a selection of a portion of a third ijpiage, said portion of said third image includes a 
subset of said set of colors; and 

receiving one or more blending values for feaid subset of said set of colors. 



71. An apparatus according to claim 69,\wherein: 
said step of comparing includes the steps of: 
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comparing a first color value for a first pixel of said second image to a color map, said 
color map stores color values for said set of colors and said blending values for said set of colors, and 
identifying a first Wending value corresponding to said first color value based on said 

color map; and \ 

said step of causing a blending includes using said first blending value as part of aprocess to 
blend said first pixel with a corresponding pixel in said first image. 

72. An apparatus according to claim 69, wherein: 
said steps of accessing and comparing include the steps of: 

accessing a first pixel in sTaid second image, said first pixel having a first color value, 
accessing additional pixels ndbiby to said first pixel in said second image, said additional 
pixels having additional color values, \ 

comparing a first color value for said first pixel to a color map, said color map stores 
color values for said set of colors and said blending values for said set of colors, 
comparing said additional color Values to said color map, 
identifying a first blending value cokesponding to said first color value based on said 

color map, \ 

identifying additional blending corresponding to said additional color values based on 

said color map, \ 

calculating an average of said first blendinAvalue and said additional blending values, 

and \ 

adjusting said average based on a weighting factor; and 
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said step of causing attending includes using said adjusted average to blend said first pixel with 
a corresponding pixel in said first image. 

73. An apparatus according to claim 69, wherein: 

said step of causing a blending includes performing a flicker filter. 

74. An apparatus according to claim 69, wherein: 

said set of colors includes a first subset of colors and a second subset of colors; 

said first subset of colors is associated with a first blending value; and 

said second subset of colors represents a taper zone having blending values ranging from at or 

near said first blending value to a second blehding value, said second blending value represents no or 

minimal blending of said first image with said second image. 

75. An apparatus according to claim 69, wherein: 
set of colors includes multiple visibN distinct colors. 

76. An apparatus according to claim 69, wherein: 

said step of accessing color data includes Accessing color data for a first portion of said second 
image, said first portion of said image corresponds in position to said first image; and 

said step of causing a blending includes causing said first image to be blended with said first 
portion of said second image and not other portibns of said second image. 

77. An apparatus according to claim 69, wherein: 
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said second image is a live 
said steps of storing, receivi 
and causing are performed during 



video image of a live event; and 

lg a first image, receiving a second image, accessing, comparing 
baid live event. 



78. An apparatus according claim 77, wherein said method further includes the step of: 
updating said blending valued for said set of colors during said live event. 

79. An apparatus, comprising: 
one or more processors; \ 

a drawing device in communication with said one or more processors; 
an output device in communication with said one or more processors; and 
at least one storage device in communication with said one or more processors, said processors 
programmed to preform a method comprising the steps of: 

receiving video, said video depicts a surface at said live event, 
receiving a graphic manually crtoted by a human operator during said live event, and 
causing a blending of said graphid with said video such that said graphic appears to be 
drawn on said surface, said step of blending befhg performed during said live event. 

80. An apparatus according to claim 79, wherein: 

prior to said step of causing a blending, at least ^portion of said video depicts at least a portion 
of said surface at a live event and a set of one or morfe objects occluding said surface; and 

said step of causing a blending includes causing a Wending of said graphic with said portion of 
said video without drawing said graphic over said objects. 

- 19- \ 
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81. An apparatus accordi lg to claim 79, wherein: 

said step of receiving a graphic in dudes receiving two dimensional position information for said 
graphic created in relation to a two di nensional image; and 
said method further includes the steps of: 

converting a first set of one or more two dimensional positions to one or more three 
dimensional locations, said first set of one or more two dimensional positions correspond to said two 
dimensional position information for said graphic, and 

converting said one or mclre three dimensional locations to a second set of one or more 
two dimensional positions, said step of cauking attending includes blending said graphic with said video 
based on said second set of one or morfe two dimensional positions. 

82. An apparatus according to claim 81, wherein: 

said two dimensional image is dart of a first video frame from a first camera; and 
said step of causing a blending includes blending said graphic with a second video frame from 
a second camera. I 

83. An apparatus according to claim 79, further comprising the step of: 
smoothing said graphic prior to said step of blending, said graphic is a drawing. 

84. An apparatus according to claim 79, wherein said step of causing a blending includes: 
adding said graphic to a first vided image at a first position in said first video image; 
adding said graphic to a second videoimage at a second position in said second video image; 
said first position is different than said second position; 

\-20- 
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said first position corresponds to a positionlof an image of a first portion of said surface in said 
first video image; and 

said second position corresponds to a positidjp of said image of said first portion of said surface 

in said second video image. 



85. An apparatus, comprising: 
one or more processors; 

a drawing device in communication with sa|d one or more processors; 
an output device in communication with said one or more processors; and 
at least one storage device in communication with said one or more processors, said processors 
programmed to preform a method comprising the steps of: 

receiving video, at least a portion of said video depicts at least a portion of a surface 
at a live event and a set of one or more objects occluding said surface, 

receiving a graphic manually created by ahuman operator during said live event, and 
causing a blending of a graphic with said portion of said video without drawing said 
graphic over said objects, said steps of receiving video, rea iving a graphic and causing are performed 
during said live event. 



86. An apparatus according to claim 85, where m 



the steps of: 

accessing color data for said video; 

using said color data to access stored blending valuesj, said stored blending values arc organized 
by colors; and 
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said step of causing a blending includes 



of: 



causing a blending of said graphic with said video based on using said blending values. 

87. An apparatus according fp claim 86, wherein said method further includes the steps 

receiving a selection of a portion of an image, said portion of said image includes a set of colors; 
receiving said blending values for said set of colors; and 
storing said blending values. 



88. An apparatus according to claim 87, wherein: 
said steps of accessing and using include the steps of: 
accessing a first pixel in said video, 

comparing a first color value for said first pixel to a color map, said color map stores 
color values and corresponding blending values, and 

identifying a first blending value corresponding to said first color value based on said 

color map; and 

said step of causing a blending includes using skid first blending value as part of a process to 
blend said first pixel with a corresponding pixel in said graphic. 



89. An apparatus according to claim 85, vsiherein: 
said step of receiving a graphic includes receiving twp dimensional position information for said 
graphic created in relation to a two dimensional image;! and 
said method further includes the steps of: 
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converting a first set of one or more two dimensional positions to one or more three 
dimensional locations, said first set of orie or more two dimensional positions correspond to said two 
dimensional position information for said graphic, and 

converting said one or moreVhree dimensional locations to a second set of one or more 
two dimensional positions, said step of blendW includes blending said graphic with said video based 
on said second set of one or more two dimensional positions. 

90. An apparatus according to claimp, wherein said method further includes the steps 

of: \ 

smoothing said graphic prior to said step df blending, said graphic is a drawing. 

91. An apparatus according to claim 85, whWein said step of causing ablending includes: 
adding said graphic to a first video image at A first position in said first video image; 
adding said graphic to a second video image at a sfecond position in said second video image; 
said first position is different than said second Position; 

said first position corresponds to a position of an image of a first portion of said surface in said 
first video image; and \ 

said second position corresponds to a position of said image of said first portion of said surface 

in said second video image. \ 

92. An apparatus, comprising: \ 
one or more processors; \ 

a drawing device in communication with said one or more processors; 
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an output device in communication with said one or more processors; and 
at least one storage device in comnkinication with said one or more processors, said processors 
programmed to preform a method comprising the steps of: 

receiving video, said video depicts a surface at said live event, 

receiving a two dimensionalposition information for at least a portion of a graphic 
created in relation to a two dimensional image, 

converting a first set of one or more two dimensional positions to one or more three 
dimensional locations, said first set of one or more two dimensional positions correspond to said two 
dimensional position information, \ 

converting said one or more three oimensional locations to a second set of one or more 
two dimensional positions, and I 

causing a blending of said graphic with said video based on said second set of one or 
more two dimensional positions. \ 

93. An apparatus according to claim 921 further comprising: 

a first set of one or more camera view sensore, said method further includes the step of 
receiving camera sensor data for a first camera from saidlfirst set of one or more camera view sensors, 
said video being captured by said first camera, said step of converting said one or more three 
dimensional locations to a second set of one or more two dimensional positions is performed based on 
said camera sensor data for said first camera. I 

94. An apparatus according to claim 93, further comprising: 
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a second set of one or more earner; l 



view sensors, said method further including the step of 



receiving camera sensor data for a second c amera from said second set of one or more camera view 
sensors, said two dimensional image being captured by said second camera, said step of converting said 
first set of one or more two dimensional positions to one or more three dimensional locations is 
performed based on said camera sensor dafca for said second camera. 



95. An apparatus according to claim 92, further comprising: 

a first set of one or more camera vievd sensors, said method further including the step of 
receiving camera sensor data for a first camera from said first set of one or more camera view sensors, 
said two dimensional image being captured by said first camera, said step of converting said first set 
of one or more two dimensional positions to onetar more three dimensional locations is performed 
based on said camera sensor data for said first camera. 

96. An apparatus according to claim 92, wherein: 

said first set of one or more two dimensional jjpsitions pertains to only a new portion of said 
graphic; and 

said one or more three dimensional locations Wertain to said graphic in its entirety. 

97. An apparatus according to claim 92, wherdin said method further includes the step of: 
smoothing said graphic prior to said step of bleiMing, said graphic is a drawing. 



98. An apparatus, comprising: 



one or more processors; 
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a drawing device in communication 
an output device in communication 



/ 

l with said one or more processors; 
with said one or more processors; and 
at least one storage device in communic ation with said one or more processors, said processors 
programmed to preform a method comprising the steps of: 

iepicts a surface at said live event, 
: i drawing manually created by a human operator during 



receiving video, said video 
receiving at least a portion of 



said live event, 



smoothing said drawing, a id 

blending said smoothed drawing with said video during said live event. 

99. An apparatus according to c aim 98, wherein said method further includes the step of: 
thickening said drawing. 



Remarks 



The amendments fix a typographical error from the word processor's spell check tool. 
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The Commissioner is authorized to charge any underpayment or credit any overpayment to 
Deposit Account No. 06- 1 325 for any matter in connection with this document, including any fee for 
extension of time which may be required. 

Respectfully submitted, 

FLffiSLER, DUBB, MEYER & LOVEJOY LLP 

febrofi/u 2Q<34£ >\ By/^^ 

^ Burt Magen Q 

Reg. No. 37,175 

FLIESLER, DUBB, MEYER & LOVEJOY LLP 
Four Embarcadero Center, Suite 400 
San Francisco, California 94111-4156 
Telephone (415) 362-3800 



Date: 
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